Automatic mirror adjustment using an in-car camera system

ABSTRACT

An apparatus comprising a sensor, an interface and a processor. The sensor may be configured to generate a video signal based on a targeted view of a driver. The interface may be configured to receive status information about one or more components of a vehicle. The processor may be configured to generate a control signal in response to a determined field of view of the driver. The control signal may be used to adjust one or more mirrors of the vehicle. The field of view may be determined based on (i) the video signal and (ii) the status information.

FIELD OF THE INVENTION

The present invention relates to video capture devices generally and,more particularly, to an automatic mirror adjustment using an in-carcamera system.

BACKGROUND OF THE INVENTION

Blind zones, or blind spots, in a vehicle are a leading cause ofaccidents between vehicles. A driver checking rear or side view mirrorsthat are not in correct alignment before changing lanes assumes, oftenincorrectly, that because there is no vehicle visible in the mirror thata lane change can safely be performed. However, a blind zone due toincorrect alignment of vehicle mirrors may cause nearby vehicles to beout of the field of view of the driver.

Correct alignment of the rear or side view mirrors based on a seatingposition, height, and distance from the mirrors of the driver couldreduce accidents due to blind zones by a significant factor.Nevertheless, studies show that a majority of drivers do not know how toalign mirrors correctly. Furthermore, when drivers use vehicles that arenot their own, or use a shared vehicle, drivers often find the processof manual mirror adjustment of vehicle mirrors to be too bothersome fora single trip.

Many vehicles are being equipped with in-car camera systems that monitorattentiveness and drowsiness of the driver. Using camera systems todetermine blind spots and to adjust vehicle mirrors may significantlyincrease driver safety.

It would be desirable to implement an automatic mirror adjustment usingan in-car camera system.

SUMMARY OF THE INVENTION

The present invention concerns an apparatus comprising a sensor, aninterface and a processor. The sensor may be configured to generate avideo signal based on a targeted view of a driver. The interface maybeconfigured to receive status information about one or more components ofa vehicle. The processor may be configured to generate a control signalin response to a determined field of view of the driver. The controlsignal may be used to adjust one or more mirrors of the vehicle. Thefield of view may be determined based on (i) the video signal and (ii)the status information.

The objects, features and advantages of the present invention includeproviding automatic mirror adjustment that may (i) determine a field ofview of a driver, (ii) adjust vehicle mirrors based on characteristicsof each driver, (iii) reduce blind zones, (iv) estimate a location ofeyes of a driver, (v) estimate a distance of a driver from a camerabased on objects of known size, (vi) adjust vehicle mirrors based oncharacteristics of a vehicle, (vii) be implemented using existing in-carcameras, (viii) warn a user that mirrors are out of alignment, (ix)provide manual fine-tuning for mirror adjustment along with automaticadjustment, (x) be activated manually, (xi) continually evaluate mirroralignment and/or (xii) be easy to implement.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will be apparent from the following detailed description andthe appended claims and drawings in which:

FIG. 1 is a block diagram of an example embodiment of an apparatus;

FIG. 2 is a block diagram of another example embodiment of an apparatus;

FIG. 3 is a diagram illustrating a capture device detecting a driver ofan automobile;

FIG. 4 is a diagram illustrating determined fields of view;

FIG. 5 is a diagram illustrating eye detection and object sizecomparison;

FIG. 6 is a flow diagram illustrating a method for automatic mirroradjustment;

FIG. 7 is a flow diagram illustrating a method for comparing objects ofknown size to determine distance; and

FIG. 8 is a flow diagram illustrating a method for notifying a driver ofmirror misalignment.

DETAILED DESCRIPTION OF EMBODIMENTS

Referring to FIG. 1, a block diagram of an apparatus 100 is shown inaccordance with an embodiment of the present invention. The apparatus100 may be a camera system. The camera system 100 may comprise a block(or circuit) 102, a block (or circuit) 104, a block (or circuit) 106,and a block (or circuit) 108. The circuit 102 may implement a capturedevice. The circuit 104 may implement an interface. The circuit 106 maybe configured as a processor. The circuit 108 may be configured as amemory. The camera system 100 is shown connected to a block (or circuit)110. The circuit 110 may be an external communication device. In someembodiments, the communication device 110 may be implemented as part ofthe camera system 100. The camera system 100 is shown receiving inputfrom a block (or circuit) 112. The block 112 may be a lens (e.g., acamera lens). In some embodiments, the lens 112 may be implemented aspart of the camera system 100. In some embodiments, the camera system100 may be implemented as a drop-in solution (e.g., installed as onecomponent).

The capture device 102 may present a signal (e.g., VIDEO) to theprocessor 106. The interface 104 may present a signal (e.g., STATUS) tothe processor 106. The processor 106 may be configured to receive thesignal VIDEO and/or the signal STATUS. The processor 106 may beconfigured to generate a signal (e.g., CONTROL). The inputs, outputsand/or arrangement of the components of the camera system 100 may bevaried according to the design criteria of a particular implementation.

Referring to FIG. 2, a block diagram of an apparatus 100′ is shown inaccordance with an embodiment of the present invention. The camerasystem 100′ may comprise the capture device 102′, the interface 104, theprocessor 106, the memory 108, the communication device 110 and/or thelens 112. The camera system 100′ may be a distributed system (e.g., eachcomponent may be implemented separately throughout the vehicle 50). Thecapture device 102′ may comprise a block (or circuit) 120 and/or a block(or circuit) 122. The circuit 120 may be a sensor. The circuit 122 maybe a processor (e.g., a processor separate from the processor 106). Thecapture device 102′ may implement a separate memory.

Referring to FIG. 3, an embodiment 150 illustrating the capture device102 detecting the driver of an automobile/vehicle 50 is shown. Thecamera system 100 is shown inside the vehicle 50. The capture device 102is shown inside the vehicle 50. A driver 152 is shown seated in thevehicle 50. A side view mirror 154 is shown attached to the vehicle 50.

The camera system 100 is shown in the rear of the vehicle 50. A locationof the camera system 100 may be varied according to the design criteriaof a particular implementation. For example, in some embodiments, thevehicle 50 may allow for installation of the camera system 100 in a rearend of the vehicle 50. In other embodiments, the vehicle 50 may allowfor installation of the camera system 100 in a front end of the vehicle50. For example, the camera system 100 may be installed near and/or withthe capture device 102 (e.g., in a dashboard of the vehicle 50). Inanother example, the camera system 100 may be distributed throughout thevehicle 50 (e.g., connections may be implemented between the camerasystem 100 and the capture device 102 such as a direct wired connectionand/or a connection using a common bus line).

Generally, a position of the side view mirror 154 is adjustable usingmanual controls in the vehicle 50 available to the driver 152 (e.g.,buttons on a panel on the inside of the door beside the driver 152,buttons on a center console, an interface on a touchscreen, buttons on asteering wheel, etc.). The camera system 100 may adjust theposition/orientation of the side view mirror 154 automatically based ona field of view of the driver 152 determined by the processor 100. Insome embodiments, the camera system 100 may adjust theposition/orientation of the side view mirror 154 and the driver 152 maymake further adjustments and/or fine-tune the positioning of the mirror154 using the manual controls.

The capture device 102 is shown detecting the driver 152. The capturedevice 102 may also detect other objects in the vehicle (e.g., a seat, ahead rest, an arm rest, a rear window, a seatbelt, a center console,etc.). Based on the detected driver 152, the processor 106 may determinea position (e.g., a distance) of the driver 152 and/or a field of viewof the driver 152.

Referring to FIG. 4, an embodiment 200 illustrating determined fields ofview is shown. The vehicle 50 is shown having mirrors 154 a-154 c. Themirror 154 a may be a side view mirror on the driver side of the vehicle50. The mirror 154 b may be a rear view mirror of the vehicle 50. Themirror 154 c may be a side view mirror on the passenger side of thevehicle 50. The number and/or types of the mirrors 154 a-154 c may bevaried according to the design criteria of a particular implementation.

Fields of view (e.g., FOVs) 202 a-202 c are shown. The FOVs 202 a-202 cmay be determined by the processor 106. The FOVs 202 a-202 c maycorrespond with the mirrors 154 a-154 c. The FOV 202 a is shown withrespect to the side view mirror 154 a. The FOV 202 b is shown withrespect to the rear view mirror 154 b. The FOV 202 c is shown withrespect to the side view mirror 154 c. The FOVs 202 a-202 c may bedetermined based on a position of the driver 152, the orientation of themirrors 154 a-154 c and/or the characteristics of the vehicle 50.

Each of the FOVs 202 a-202 c may represent a range of view when thedriver 152 looks at a corresponding one of the mirrors 154 a-154 c.Generally, the FOVs 202 a-202 c provide a view behind and/or beside thevehicle 50. The FOVs 202 a-202 c may be arranged such that blind spotsand/or blind zones are eliminated, reduced and/or minimized. Arrangingthe FOVs 202 a-202 c to reduce blind spots/zones may improve safety whenchanging lanes.

Each of the mirrors 154 a-154 c is shown connected to the camera system100. The capture device 102 is shown connected to the camera system 100.The mirrors 154 a-154 c may send status information to the camera system100 (e.g., to the interface 104). The camera system 100 may send thesignal CONTROL to each of the mirrors 154 a-154 c to adjust thepositioning and/or orientation.

The camera system 100 may be implemented to calculatepositioning/orientation of the mirrors 154 a-154 c for the vehicle 50(e.g., a car, a truck, a motorcycle and/or any type of automobile). Thecalculated positioning/orientation of the mirrors 154 a-154 c may bebased on the FOVs 202 a-202 c of the driver 152 of the vehicle 50. Thecamera system 100 may determine the corresponding FOVs 202 a-202 c foreach of the mirrors 154 a-154 c of the vehicle 50. A number of FOVs 202a-202 c may be determined (e.g., one for each of the mirrors 154 a-154 cof the vehicle 50). For example, the vehicle 50 may have three mirrors(e.g., the two side view mirrors 154 a and 154 c and the rear viewmirror 154 b) and there may be three corresponding FOVs 202 a-202 cdetermined by the camera system 100. The camera system 100 may interfacewith other systems of the automobile 50 to align each of the mirrors 154a-154 c (e.g., each mirror may be aligned independently) automaticallyfor each driver. The mirrors 154 a-154 c may be positioned to reduceand/or eliminate blind spots and/or blind zones of the driver 152. Forexample, the positioning of the mirrors 154 a-154 c may be calculatedbased on characteristics of the driver 152 (height, preferences, etc.)and/or characteristics of the vehicle 50 (size, obstructions, visiblezones, etc.).

In some embodiments, the camera system 100 may be installed in thevehicle 50 at a time of manufacturing. For example, the camera system100 may be installed on a particular type (e.g., model, make, year,etc.) of vehicle 50 and the camera system 100 may store pre-determinedstatus information about the vehicle 50 (e.g., a size, seat positioning,range of view of the mirrors, known sizes of particular objects, etc.).

In some embodiments, the camera system 100 may be installed in thevehicle 50 as a separate component (e.g., an after-market part). In oneexample, the camera system 100 may be designed and/or sold for aparticular make/model of the vehicle 50 and store pre-determined statusinformation (e.g., in the memory 108). In another example, the camerasystem 100 may be programmable and the status information may be enteredin the camera system 100 based on the status information of the vehicle50. For example, an online database may be implemented with statusinformation for various types of vehicles (e.g., make, model, year,etc.) and the status information may be downloaded and stored in thecamera system 100. The implementation of the camera system 100 in thevehicle 50 and/or a method of storing information about the vehicle 50may be varied according to the design criteria of a particularimplementation.

The capture device 102 may capture video image data (e.g., from the lens112). In some embodiments, the capture device 102 may be a videocapturing device such as a camera. In some embodiments, the capturedevice 102 may be a component of a camera (e.g., a camera pre-installedin the vehicle 50). The capture device 102 may capture data receivedthrough the lens 112 to generate a bitstream (e.g., generate videoframes). For example, the capture device 102 may receive light from thelens 112. The lens 112 may be directed at the driver 152 to provide atargeted view of the driver 152. The capture device 102 may transformthe received light into digital data (e.g., a bitstream). In someembodiments, the capture device 102 may perform an analog to digitalconversion. For example, the capture device 102 may perform aphotoelectric conversion of the light received by the lens 112. Thecapture device 102 may transform the bitstream into video data, a videofile and/or video frames (e.g., perform encoding). For example, thevideo data may be a digital video signal. The digital video signal maycomprise video frames (e.g., sequential digital images).

The video data of the driver 152 may be represented as thesignal/bitstream/data VIDEO (e.g., a digital video signal). The capturedevice 102 may present the signal VIDEO to the processor 106. The signalVIDEO may represent the video frames/video data. The signal VIDEO may bea video stream captured by the capture device 102. In some embodiments,the capture device 102 may be implemented in the camera. In someembodiments, the capture device 102 may be configured to add to existingfunctionality of the camera.

The capture device 102 may be installed in the vehicle 50 (e.g., in theinterior of the car 50 directed at the driver 152). In some embodiments,the capture device 152 may be pre-installed in the vehicle 50 and thecamera system 100 may connect to the capture device 102. In otherembodiments, the capture device 102 may be part of the camera system100. The capture device 102 may be configured for driver monitoring. Forexample, the capture device 102 may be implemented to detect drowsinessand/or attentiveness of the driver 152. In another example, the capturedevice 152 may record the driver 152, (e.g., for use inteleconferencing). The capture device 102 may be configured to recognizethe driver 152 through facial recognition. The camera system 100 may beconfigured to leverage pre-existing functionality of the pre-installedcapture device 102. The implementation of the capture device 102 may bevaried according to the design criteria of a particular implementation.

In some embodiments, the capture device 102′ may implement the sensor120 and/or the processor 122. The sensor 120 may receive light from thelens 112 and transform the light into digital data (e.g., thebitstream). For example, the sensor 120 may perform a photoelectricconversion of the light from the lens 112. The processor 122 maytransform the bitstream into a human-legible content (e.g., video data).For example, the processor 122 may receive pure (e.g., raw) data fromthe sensor 120 and generate (e.g., encode) video data based on the rawdata (e.g., the bitstream). The capture device 102′ may have a memory tostore the raw data and/or the processed bitstream. For example, thecapture device 102′ may implement a frame memory and/or buffer to store(e.g., provide temporary storage and/or cache) one or more of the videoframes (e.g., the digital video signal). The processor 122 may performanalysis on the video frames stored in the memory/buffer of the capturedevice 102′.

In some embodiments the capture device 102′ may be configured todetermine a location of the eyes of the driver 152. For example, theprocessor 122 may analyze the captured bitstream (e.g., using machinevision processing), determine a location of the eyes of the driver 152and present the signal VIDEO (e.g., comprising information about thelocation of the eyes of the driver 152) to the processor 106. Theprocessor 122 may be configured to determine the location of the eyes ofthe driver 152 (e.g., less analysis is performed by the processor 106).In another example, the processor 122 may generate the signal VIDEOcomprising video frames and the processor 106 may analyze the videoframes to determine the location of the eyes of the driver (e.g., moreanalysis is performed by the processor 106). The analysis performed bythe processor 122 and/or the processor 106 may be varied according tothe design criteria of a particular implementation.

The interface 104 may receive data from one or more components of thevehicle 50. The signal STATUS may be generated in response to the datareceived from the components of the vehicle 50. In some embodiments, theinterface 104 may receive data from the processor 106. The interface 104may send data (e.g., instructions) from the processor 106 to thecomponents of the vehicle 50. The data from the components of thevehicle 50 may be a seat position, a seat recline position, an angle ofthe bottom seat cushion, a mirror orientation, a speed of the vehicle,any information available from an on-board diagnostics (OBD) port of thevehicle 50, etc. The type of data and/or the number of components of thevehicle 50 that provide data may be varied according to the designcriteria of a particular implementation.

The processor 106 may be configured to execute computer readable codeand/or process information. The processor 106 may be configured toreceive input and/or present output to the memory 108. The processor 106may be configured to present and/or receive other signals (not shown).The number and/or types of inputs and/or outputs of the processor 106may be varied according to the design criteria of a particularimplementation.

In some embodiments, the processor 106 may receive the signal VIDEO fromthe capture device 102 and detect the driver 152 in the video frame. Insome embodiments, the processor 122 may be configured to detect thedriver 152 and the processor 106 may receive the location of the eyes ofthe driver from the capture device 102′. In some embodiments, theprocessor 106 may be configured to analyze the video frame (e.g., thesignal VIDEO). The processor 106 may be configured to detect a locationand/or position of the eyes of the driver in the video frame. Theprocessor 106 may determine a distance of the eyes of the driver fromthe camera based on information from the signal STATUS. In someembodiments, the processor 106 may receive the location of the eyes fromthe capture device 102′ and distance of the eyes of the driver from thecamera through the interface 104. The information received by theprocessor 106 and/or the analysis performed by the processor 106 may bevaried according to the design criteria of a particular implementation.

Based on the distance and/or location of the eyes of the driver and/orthe mirrors of the vehicle, the processor 106 may determine the FOVs 202a-202 c of the driver 152. The FOVs 202 a-202 c of the driver 152 may bebased on the signal VIDEO and/or the signal STATUS. The processor 106may generate the signal CONTROL in response to the determined FOVs 202a-202 c.

The signal CONTROL may be implemented to provide instructions to thevarious components of the vehicle 50. For example, the signal CONTROLmay be used by the mirrors 154 a-154 c to adjust an orientation of themirrors 154 a-154 c (e.g., based on the determined FOVs 202 a-202 c).The orientation of the mirrors 154 a-154 c may be an angle, a locationand/or a position (e.g., any characteristic of the mirrors 154 a-154 cthat may affect the FOVs 202 a-202 c of the driver 152). In someembodiments, the signal CONTROL may be presented to the interface 104and the interface 104 may pass the signal CONTROL to one of thecomponents of the vehicle 50. In some embodiments, the signal CONTROLmay be presented directly to one of the components of the vehicle 50 bythe processor 106.

The processor 106 and/or the processor 122 may be implemented as anapplication specific integrated circuit (e.g., ASIC) or asystem-on-a-chip (e.g., SOC). The processor 106 and/or the processor 122may be configured to determine a current size of an object of known size(e.g., an object having a reference size). The processor 106 and/or theprocessor 122 may detect an object of known size in each video frame.The processor 106 and/or the processor 122 may determine a number ofpixels (e.g., a width and/or a height) comprising the object of knownsize in the video frame. Based on the number of pixels of the object ofknown size in the video frame, the processor 106 and/or the processor122 may estimate a distance of the driver from the lens 112. Whether thedetection of the object of known size is performed by the processor 106and/or the processor 122 may be varied according to the design criteriaof a particular implementation.

The memory 108 may store data. The memory 108 may be implemented as acache, flash memory, DRAM memory, etc. The type and/or size of thememory 108 may be varied according to the design criteria of aparticular implementation. The data stored in the memory 108 maycorrespond to the objects of known size. For example, the memory 108 maystore a reference size (e.g., the number of pixels of the object ofknown size in a video frame at a known distance) of the object of knownsize. The reference size stored in the memory 108 may be used to comparethe current size of the object of known size detected in a current videoframe. The comparison of the size of the object of known size in thecurrent video frame and the reference size may be used to estimate adistance of the driver from the lens 112.

The memory 108 may store the pre-determined status information of thevehicle 50. For example, the status information of the vehicle 50 may beupdated by over-writing the status information stored in the memory 108.In some embodiments, the memory 108 may store pre-defined preferences(e.g., mirror orientation) for each driver.

The communication device 110 may send and/or receive data to/from theinterface 104. In some embodiments, the communication device 110 may bethe OBD of the vehicle 50. In some embodiments, the communication device110 may be implemented as a satellite (e.g., a satellite connection to aproprietary system). For example, the satellite 110 may receive datafrom one or more vehicles. The data received by the satellite 110 may beused by vehicle manufacturers to improve the driving experience and/ordetect problems with vehicles. The data received by the satellite 110may be used to provide roadside assistance. For example, aggregate datafrom the communication device 110 may determine behavioral patterns ofdrivers (e.g., how often drivers check the mirrors, whether driverschange the orientation of the mirrors, attentiveness of drivers, etc.).

The communication device 110 may implement vehicle-to-vehiclecommunication. In some embodiments, the communication device 110 mayimplement a wireless and/or cellular communication system (e.g., a 4GLTE connection). In some embodiments, the communication device 110 mayprovide a connection to a device of the driver 152 (e.g., a Bluetoothconnection to a smartphone, a ZigBee connection to a mesh network ofinterconnected devices, a Wi-fi connection to a tablet computing device,etc.). The implementation of the communication device 110 may be variedaccording to the design criteria of a particular implementation.

The lens 112 (e.g., a camera lens) may be directed at the driver 152(e.g., directed at the seat of the driver 152, provide a targeted viewof the driver 152, etc.). For example, the lens 112 may be mounted on adashboard of the vehicle 50. In another example, the lens 112 may bemounted as part of a console of the vehicle 50. The lens 112 may beaimed to capture environmental data (e.g., light). The lens 112 may beconfigured to capture and/or focus the light for the capture device 102.Generally, the sensor 120 is located behind the lens 112. Based on thecaptured light from the lens 112, the capture device 102 may generate abitstream and/or video data.

Referring to FIG. 5, video frames 250 and 250′ illustrating eyedetection and object size comparison are shown. The video frames 250 and250′ may be video frames generated by the capture device 102. The videoframes 250 and/or 250′ may represent a targeted view captured by thelens 112 mounted on the dashboard of the vehicle 50. The view capturedfor each of the video frames may be varied according to the designcriteria of a particular implementation.

The video frame 250 may represent a reference frame. For example, thereference frame 250 may be stored in the memory 108. The reference frame250 shows an object of known size 252 in the vehicle 50. The object ofknown size 252 may be a head rest of the driver side seat. The processor106 and/or the processor 122 may determine the width of the object ofknown size 252 (e.g., based on the number of pixels in the video frame).The memory 108 may store the width of the object of known size 252(e.g., D_REF). The width D_REF may be determined when the object ofknown size 252 is at a known distance from the lens 112. The width D_REFmay be stored in the memory 108 as the reference size.

The video frame 250′ may represent a current frame. For example, thecapture device 102 may send the signal VIDEO as the current frame to theprocessor 106. In another example, the processor 122 may generate and/oranalyze the current frame 250′ (e.g., the current frame 250′ may bestored in a memory of the capture device 102′) and send a result of theanalysis (e.g., the location of the eyes of the driver 152) to theprocessor 106. The current frame 250′ shows the vehicle 50′, the driver152 and an object of known size 252′. The current frame 250′ may beanalyzed by the processor 106 and/or the processor 122. The processor106 and/or the processor 122 may detect the eyes of the driver 152.Boxes 254 a-254 b may represent the detected eyes of the driver 152 inthe current frame 250′.

The object of known size 252 and/or 252′ may be an object physicallyconnected to the seat of the driver 152. For example, the object ofknown size 252 and/or 252′ may be the head rest as shown. In someembodiments, the object of known size 252 and/or 252′ may be the seat ofthe driver 152, an arm rest of the seat of the driver 152 and/or a seatbelt. Other objects in the reference frame 250 and/or the current frame250′ may have a known size (e.g., a steering wheel, a rear seat, adashboard, a sunroof, a moonroof, etc.) but may be unsuitable fordetermining a distance of the driver 152 from the lens 112 (e.g.,objects that generally have a fixed position in the video frames).

The distance of the driver 152 from the lens 112 may be estimated basedon the object of known size 252 and/or 252′ and characteristics of thedriver 152. For example, if the headrest 252′ is determined to be 4.5feet away from the lens 112 an average sitting posture and head size maybe used to estimate that the eyes of the driver 152 may be 3.5 feet fromthe lens 112. The characteristics of the driver 152 and/or theestimations performed may be varied according to the design criteria ofa particular implementation.

The processor 106 and/or the processor 122 may be configured to detectthe object of known size 252′ in the current frame 250′. The object ofknown size 252′ may be the head rest. The head rest 252′ is shown closerin the current frame 250′ than the head rest 252 in the reference frame250. The processor 106 and/or the processor 122 may determine the widthof the object of known size 252′ (e.g., the number of pixels in thevideo frame). The memory 108 may store the width of the object of knownsize 252′ (e.g., D_CURRENT). The width D_CURRENT may be used as thecurrent size of the object of known size 252′. The current sizeD_CURRENT may be compared to the reference size D_REF by the processor106 and/or the processor 122. Based on the comparison of the currentsize D_CURRENT and the reference size D_REF, the processor 106 and/orthe processor 122 may estimate a distance of the driver 152 from thelens 112.

Using the detected eyes 254 a-254 b and the estimated distance of thedriver 152 from the lens 112, the processor 106 and/or the processor 122may determine the position (e.g., 3D coordinates and/or locationcoordinates) of the eyes of the driver 152. For example, the location ofthe detected eyes 254 a-254 b may represent one coordinate (e.g., alocation coordinate on a first axis) for a vertical location of each eyein 3D space, and one coordinate (e.g., a location coordinate on a secondaxis) for a horizontal location of each eye in 3D space. The determineddistance from the lens 112 may represent one coordinate (e.g., alocation coordinate on a third axis) for a depth location of each of thedetected eyes 254 a-254 b in 3D space. For example, the processor 122may determine the location of the detected eyes 254 a-254 b in 3D spaceand transmit the location (e.g., using the signal VIDEO) to theprocessor 106.

Based on the determined position/location of the eyes of the driver 152and the orientation of the mirrors 154 a-154 c (e.g., determined basedon the signal STATUS), the processor 106 may determine the FOVs 202a-202 c. The processor 106 may determine whether the FOVs 202 a-202 cshould be adjusted to reduce and/or eliminate blind spots and/or blindzones. The processor 106 may generate the signal CONTROL in response toone or more of the determined FOVs 202 a-202 c. The signal CONTROL mayprovide instructions for adjusting the mirrors 154 a-154 c.

Referring to FIG. 6, a method (or process) 300 is shown. The method 300may perform automatic mirror adjustment. The method 300 generallycomprises a step (or state) 302, a decision step (or state) 304, a step(or state) 306, a step (or state) 308, a step (or state) 310, a step (orstate) 312, a step (or state) 314, a decision step (or state) 316, astep (or state) 318, and a step (or state) 320.

The state 302 may start the method 300. Next, the method 300 may move tothe decision state 304. If the decision state 304 determines not toinitiate a mirror adjustment, the method 300 may move to the state 306.The state 306 may wait for a manual command. Next, the method 300 mayreturn to the decision state 304. If the decision state 304 determinesto initiate a mirror adjustment, the method 300 may move to the state308.

The state 308 may capture a video signal and/or video data (e.g., VIDEO)of the driver 152. Next, the state 310 may determine 3D coordinates ofthe eyes of the driver 152. The state 312 may retrieve statusinformation (e.g., STATUS) of the vehicle components. The state 314 maydetermine the field of view (e.g., the FOVs 202 a-202 c) of the driver152 based on the 3D coordinates and/or the status information. Next, themethod 300 may move to the decision state 316.

If the decision state 316 determines not to adjust the mirrors 154 a-154c, the method 300 may move to the state 306. If the decision state 316determines to adjust the mirrors 154 a-154 c, the method 300 may move tothe state 318. The state 318 may adjust the mirrors 154 a-154 c (e.g.,using the signal CONTROL) based on the determined FOVs 202 a-202 c.Next, the method 300 may move to the state 320, which ends the method300.

The camera system 100 may determine a location and/or position of eyesof the driver 152 with respect to the mirrors 154 a-154 c. For example,the location/position of the eyes may be represented in 3D coordinates.In another example, the location/position of the eyes may be representedas a vector. In yet another example, displacement of the eyes withrespect to the mirrors 154 a-154 c may be determined. The method ofdetermining the distance and/or position of the eyes of the driver 152may be varied according to the design criteria of a particularimplementation. The camera system 100 may determine an orientation ofeach of the mirrors 154 a-154 c (e.g., based on the signal STATUS).Based on the orientation of the mirrors 154 a-154 c and/or theposition/location of the eyes of the driver 152 a FOV (e.g., the FOVs202 a-202 c) for each eye for each mirror relative to the vehicle 50 maybe determined.

The location/position of the eyes of the driver 152 may be determinedusing a camera (e.g., the lens 112) facing the driver 152. The cameralens 112 may present a signal to the capture device 102. In someembodiments, the capture device 102 may present the signal VIDEO to theprocessor 106. The processor 106 may determine and/or recognize the eyesof the driver 152. The processor 106 may locate the eyes of the driver152 in each video frame (e.g., the signal VIDEO). Based on the locationof the eyes, the processor 106 may estimate the distance of the eyes ofthe driver with respect to the camera lens 112. In other embodiments,the capture device 102′ may determine and/or recognize the eyes of thedriver 152, determine a distance of the eyes of the driver 152 from thecamera lens 112 and/or present the signal VIDEO having the 3Dcoordinates of the location of the eyes of the driver 152 to theprocessor 106.

A location of the eyes in the video frame alone may not provide enoughinformation to determine the FOVs 202 a-202 c of the driver 152 and/or adistance from the camera lens 112. In some embodiments, statusinformation may be used to determine the FOVs 202 a-202 c of the driver152. For example, a position of a seat (e.g., height, depth, recline,angle of a bottom cushion of the seat, etc.) of the driver 152 may beused to determine the distance of the driver 152. The position of theseat of the driver may be available through the OBD port of the vehicle50 (e.g., for vehicles with electronic seat adjustment). In anotherexample, the current position of the mirrors 154 a-154 c may be thestatus information. The FOVs 202 a-202 c of the driver 152 may depend onthe angle of the mirrors 154 a-154 c. In yet another example, the amountof recline of the seat may be the status information. The amount ofrecline may be used to determine an angle of the eyes of the driver 152with respect to the mirrors 154 a-154 c (e.g., the rear view mirror 154b may provide a FOV showing the ceiling of the vehicle if the seat ofthe driver is reclined and the driver is looking upwards at the rearview mirror).

Referring to FIG. 7, a method (or process) 350 is shown. The method 350may compare objects of known size to determine distance. The method 350generally comprises a step (or state) 352, a decision step (or state)354, a step (or state) 356, a step (or state) 358, a step (or state)360, a step (or state) 362, and a step (or state) 364.

The state 352 may start the method 350. Next, the method 350 may move tothe decision state 354. If the decision state 354 determines the videosignal does not have a known object (e.g., the head rest, the seat, thearm rest, etc.) in the video frame, the method 350 may move to the state356. The state 356 may end the method 350. If the decision state 354determines the video signal does have the known object in the videoframe, the method 350 may move to the state 358.

The state 358 may determine the size of the known object from the videosignal/frame (e.g., a number of pixels). Next, the state 360 may comparea pixel size of the known object from the video signal/frame (e.g.,D_CURRENT) with a stored known object size (e.g., the reference sizeD_REF stored in the memory 108). The state 362 may estimate the distance(e.g., from the camera lens 112) of the known object based on thecomparison. The state 364 may estimate the distance of the eyes of thedriver 152 from the camera lens 112 based on the estimated distance ofthe known object. Next, the method 350 may move to the end state 356.

In some embodiments, a calculation using an object of known size may beperformed. For example, an object of known size may be a part of theseat of the driver 152 (e.g., the head rest, the width of the seat at agiven height, an arm rest, a seat belt, etc.). The type of object ofknown size may be varied according to the design criteria of aparticular implementation.

Knowing the width of the object, the exact FOV of the driver-facingcamera lens 112 and the number of pixels the object occupies in thevideo frame, the distance and/or position of the eyes of the driver 152from the camera lens 112 may be calculated. For example, a distance ofthe seat of the driver 152 from the camera lens 112 may be determined.Based on the distance of the seat, the distance of the eyes of thedriver 152 from the camera lens 112 may be calculated (or estimated).For example, the distance of the eyes of the driver 152 from the cameralens 112 may be estimated based on assumptions about typical sittingposture of a driver. In some embodiments, anatomical, ergonomic and/orphysiological data may be used to make assumptions about the seatingposition of the driver 152.

Referring to FIG. 8, a method (or process) 380 is shown. The method 380may notify the driver 152 of mirror misalignment. The method 380generally comprises a step (or state) 382, a decision step (or state)384, a step (or state) 386, a step (or state) 388, a step (or state)390, a decision step (or state) 392, a step (or state) 394, a step (orstate) 396, and a step (or state) 398.

The state 382 may start the method 380. Next, the method 380 may move tothe decision state 384. If the decision state 384 determines not toinitiate the automatic adjustment, the method 380 may move to the state386. The state 386 may wait a pre-determined amount of time. Next, themethod 380 may return to the decision state 384. If the decision state384 determines to initiate the automatic adjustment, the method 380 maymove to the state 388.

The state 388 may determine the mirror geometry for the vehicle 50(e.g., based on the characteristics of the vehicle 50 and/or the statusinformation stored in the memory 108). The state 390 may evaluate thefield of view of the driver 152. Next, the method 380 may move to thedecision state 392.

If the decision state 392 determines the mirror alignment cannot beimproved, the method 380 may move to the state 386. If the decisionstate 392 determines the mirror alignment can be improved, the method380 may move to the state 394. The state 394 may notify the driver 152of the mirror misalignment. The state 396 may allow the driver 152 tomanually re-calibrate the mirror alignment. Next, the state 398 may endthe method 380.

The camera system 100 may be configured to automatically adjust themirrors 154 a-154 c. The automatic adjustment process may be activatedthrough a separate button, option and/or command. The mirrors 154 a-154c may be adjusted manually for fine-tuning. In some embodiments, theautomatic adjustment process may be used in conjunction with manualadjustment of the mirrors 154 a-154 c. For example, the automaticadjustment process may adjust the mirrors 154 a-154 c to a positionbased on the determined FOVs 202 a-202 c (e.g., a position considered tobe safe for driving) and the driver 152 may then make furtheradjustments based on individual preferences.

The processor 106 may perform the automatic adjustment continuously,periodically and/or at fixed intervals (e.g., based on a pre-determinedamount of time). For example, the processor 106 may evaluate a currentalignment of the mirrors 154 a-154 c. If the alignment is determined tobe unsafe (e.g., sub-optimal, the mirror position is determined to betoo far off, etc.) the driver 152 may be notified. For example, thedriver may be notified by a warning and/or an alert (e.g., an LEDindicator, an image blinking, sound, haptic feedback from the seat,etc.). The notification may prompt the driver 152 to adjust the mirrororientation and/or change seat position.

In some embodiments, a notification may indicate that the camera system100 is automatically making adjustments to the alignment of the mirrors154 a-154 c (e.g., without confirmation from the driver). In someembodiments, the processor 106 may perform the automatic adjustment inresponse to the driver 152 adjusting the orientation of the mirrors 154a-154 c automatically and/or based on updated status information (e.g.,the driver 152 changed the position of the driver seat). In someembodiments, the automatic adjustment of the mirrors 154 a-154 c may beinitiated in response to a manual activation (e.g., the driver 152presses a button to initiate the automatic adjustment of the mirrors 154a-154 c).

In some embodiments, a direct distance measurement of the driver fromthe camera lens 112 may be determined using a depth sensing camerasystem. The camera lens 112 (or camera type) connected to the capturedevice 102 may be a depth sensing camera configured to implement one ormore of various depth sensing camera technologies. For example, thedepth sensing camera technology implemented may be time-of-flight. Inanother example, the depth sensing camera technology implemented may bestereo triangulation. In yet another example, the depth sensing cameratechnology implemented may be sheet of light triangulation, structuredlight, interferometry and/or coded aperture. In some embodiments morethan one camera lens 112 may be implemented. The type, number ofcameras, camera lenses 112 and/or depth sensing technologies implementedmay be varied according to the design criteria of a particularimplementation.

Based on the location of the eyes in the video frame and the distance ofthe eyes from the camera lens 112, an estimate of the position of theeyes (e.g., in 3D space) relative to the mirrors 154 a-154 c may bedetermined. Based on the position of the eyes relative to the mirrors154 a-154 c, the corresponding FOVs 202 a-202 c of the driver 152 foreach eye in each mirror may be determined. The camera system 100 maythen determine possible FOVs 202 a-202 c for each eye for possiblemirror orientations.

The processor 106 may be configured to determine an orientation of themirrors 154 a-154 c based on the possible mirror orientations and/orpossible FOVs 202 a-202 c. For example, the processor 106 may determinea FOV that reduces and/or eliminates blind spots and/or blind zonesbased on geometry of the mirrors 154 a-154 c and/or specifications ofthe vehicle 50. The signal CONTROL may be generated by the processor 106to provide automatic adjustment of the mirrors.

The functions performed by the diagrams of FIGS. 6-8 may be implementedusing one or more of a conventional general purpose processor, digitalcomputer, microprocessor, microcontroller, RISC (reduced instruction setcomputer) processor, CISC (complex instruction set computer) processor,SIMD (single instruction multiple data) processor, signal processor,central processing unit (CPU), arithmetic logic unit (ALU), videodigital signal processor (VDSP) and/or similar computational machines,programmed according to the teachings of the specification, as will beapparent to those skilled in the relevant art(s). Appropriate software,firmware, coding, routines, instructions, opcodes, microcode, and/orprogram modules may readily be prepared by skilled programmers based onthe teachings of the disclosure, as will also be apparent to thoseskilled in the relevant art(s). The software is generally executed froma medium or several media by one or more of the processors of themachine implementation.

The invention may also be implemented by the preparation of ASICs(application specific integrated circuits), Platform ASICs, FPGAs (fieldprogrammable gate arrays), PLDs (programmable logic devices), CPLDs(complex programmable logic devices), sea-of-gates, RFICs (radiofrequency integrated circuits), ASSPs (application specific standardproducts), one or more monolithic integrated circuits, one or more chipsor die arranged as flip-chip modules and/or multi-chip modules or byinterconnecting an appropriate network of conventional componentcircuits, as is described herein, modifications of which will be readilyapparent to those skilled in the art(s).

The invention thus may also include a computer product which may be astorage medium or media and/or a transmission medium or media includinginstructions which may be used to program a machine to perform one ormore processes or methods in accordance with the invention. Execution ofinstructions contained in the computer product by the machine, alongwith operations of surrounding circuitry, may transform input data intoone or more files on the storage medium and/or one or more outputsignals representative of a physical object or substance, such as anaudio and/or visual depiction. The storage medium may include, but isnot limited to, any type of disk including floppy disk, hard drive,magnetic disk, optical disk, CD-ROM, DVD and magneto-optical disks andcircuits such as ROMs (read-only memories) RAMs (random accessmemories), EPROMs (erasable programmable ROMs), EEPROMs (electricallyerasable programmable ROMs), UVPROM (ultra-violet erasable programmableROMs), Flash memory, magnetic cards, optical cards, and/or any type ofmedia suitable for storing electronic instructions.

The elements of the invention may form part or all of one or moredevices, units, components, systems, machines and/or apparatuses. Thedevices may include, but are not limited to, servers, workstations,storage array controllers, storage systems, personal computers, laptopcomputers, notebook computers, palm computers, personal digitalassistants, portable electronic devices, battery powered devices,set-top boxes, encoders, decoders, transcoders, compressors,decompressors, pre-processors, post-processors, transmitters, receivers,transceivers, cipher circuits, cellular telephones, digital cameras,positioning and/or navigation systems, medical equipment, heads-updisplays, wireless devices, audio recording, audio storage and/or audioplayback devices, video recording, video storage and/or video playbackdevices, game platforms, peripherals and/or multi-chip modules. Thoseskilled in the relevant art(s) would understand that the elements of theinvention may be implemented in other types of devices to meet thecriteria of a particular application.

While the invention has been particularly shown and described withreference to the preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade without departing from the scope of the invention.

The invention claimed is:
 1. An apparatus comprising: a sensorconfigured to generate a video signal based on a targeted view of adriver; an interface configured to receive status information about oneor more components of a vehicle; and a processor configured to generatea control signal in response to a determined field of view of saiddriver, wherein (A) said control signal is used to adjust one or moremirrors of said vehicle, (B) said field of view is determined based on(i) location coordinates for eyes of said driver and (ii) an orientationof said one or more mirrors, (C) said location coordinates for said eyesare determined based on analysis of one or more video frames of saidvideo signal to determine (i) one or more of a first location coordinateand a second location coordinate of said eyes and (ii) a depthcoordinate representing a distance of said eyes from said sensor, and(D) said depth coordinate is determined based on a comparison of areference number of pixels of an object in a reference video frame to acurrent number of pixels of said object in said video frames.
 2. Theapparatus according to claim 1, wherein said status informationcomprises said orientation of said one or more mirrors.
 3. The apparatusaccording to claim 1, wherein said status information comprises aposition of a seat of said driver.
 4. The apparatus according to claim3, wherein said position of said seat of said driver comprises a height,depth, recline and an angle of a bottom cushion of said seat.
 5. Theapparatus according to claim 1, wherein said interface is configured toconnect to a communication device.
 6. The apparatus according to claim5, wherein said communication device is at least one of a satellite, anon-board diagnostics port of said vehicle and a cellular system.
 7. Theapparatus according to claim 1, wherein said sensor is implemented asone of (i) a component of a camera and (ii) a video capturing device. 8.The apparatus according to claim 1, wherein said reference number ofpixels for said object is based on an object of known size.
 9. Theapparatus according to claim 8, wherein said reference number of pixelsis determined when said object of known size is at a known distance fromsaid sensor.
 10. The apparatus according to claim 1, wherein saidcontrol signal is configured to generate a warning.
 11. The apparatusaccording to claim 1, wherein said adjustment is performed based on (i)a manual activation in a first mode, (ii) a pre-determined amount oftime in a second mode and (iii) a change of said status information in athird mode.
 12. The apparatus according to claim 1, wherein said sensoris configured to implement a depth sensing camera system for determiningsaid depth coordinate.
 13. The apparatus according to claim 1, whereinsaid (i) adjustment of said mirrors is performed automatically and (ii)manual controls for said mirrors are available for manual adjustment ofsaid mirrors.
 14. The apparatus according to claim 1, wherein saiddetermined field of view of said driver comprises a number of fields ofview, each corresponding to one of a plurality of mirrors of saidvehicle.
 15. The apparatus according to claim 14, wherein said number ofsaid fields of view is three.
 16. The apparatus according to claim 1,further comprising a second processor configured to (i) receive abitstream generated by said sensor and (ii) transform said bitstreaminto said video signal of said driver.
 17. The apparatus according toclaim 1, wherein said video signal comprises a plurality of digitalimages.
 18. An apparatus comprising: a sensor configured to generate abitstream based on a targeted view implemented by a video capturedevice, wherein said targeted view is selected to capture a driver of avehicle; a first processor configured to (i) receive said bitstream fromsaid sensor and (ii) generate a video signal based on said bitstream; aninterface configured to receive status information about one or morecomponents of a vehicle; and a second processor configured to generate acontrol signal in response to a determined field of view of said driver,wherein (A) said control signal is used to adjust one or more mirrors ofsaid vehicle, (B) said field of view is determined based on (i) locationcoordinates for eyes of said driver and (ii) an orientation of said oneor more mirrors, (C) said location coordinates for said eyes aredetermined based on analysis of one or more video frames of said videosignal to determine (i) one or more of a first location coordinate and asecond location coordinate of said eyes and (ii) a depth coordinaterepresenting a distance of said eyes from said sensor, and (D) saiddepth coordinate is determined based on a comparison of a referencenumber of pixels of an object in a reference video frame to a currentnumber of pixels of said object in said video frames.
 19. The apparatusaccording to claim 18, wherein said analysis of said video frames fromsaid video signal is performed by (i) said first processor in a firstmode and (ii) said second processor in a second mode.